1. Field of the Invention
The present invention relates to an image processing apparatus and image processing method, and a recording medium storing a program for causing this image processing method to be implemented in a computer, and more particularly, it relates to an image processing apparatus, image processing method and storage medium, whereby images can be displayed without significant distortion, even in cases where updating of texture data in a particular scene is not completed in time.
2. Description of the Related Art
A game apparatus or simulation apparatus generates and displays images corresponding to the progress of a game, or the like, by executing a game program or simulation program in response to operational inputs made by an operator. For this purpose, it comprises an integral image processing apparatus and display apparatus.
The image processing apparatus comprises a geometry processing unit, rendering unit, texture processing unit, frame buffer and digital/analogue (D/A) converting unit, and the like. The geometry processing unit performs co-ordinate conversion of polygon data constituting objects. More specifically, firstly, the geometry processing unit converts polygon data from an object co-ordinates system (local co-ordinates system defining object) to a global co-ordinates system (co-ordinates system wherein objects have positional relationships). Thereupon, the geometry processing unit converts polygon data from a three-dimensional (3D) global co-ordinates system to a two-dimensional (2D) screen co-ordinates system (co-ordinates system defined on a screen). In other words, the geometry processing unit carries out perspective processing.
The rendering unit converts polygon data that has been converted to the screen co-ordinates system into image data in pixel units. The texture processing unit has a texture memory storing texture data (colours, patterns, and the like, of the surfaces of objects), and it performs texture mapping.
The image data in pixel units generated by the rendering unit is stored in the frame buffer. The digital/analogue converting unit converts the digital-format image data in pixel units stored in the frame buffer into an analogue signal, which is supplied to the display apparatus, where images are displayed.
When displaying images, high speed is required in the rendering process. Consequently, it is also necessary to access the texture memory at high speed. In general, in order to perform high-speed access, it is desirable to use an SRAM (Static Random Access Memory) as a texture memory, but compared to a DRAM (Dynamic Random Access Memory), or the like, an SRAM involves extremely high cost per unit bit (high cost with respect to capacity).
On the other hand, in order to make the displayed images more realistic, it is necessary to use more texture data, and in cases where a DRAM is used for the texture memory, due to the requirement for high-speed access to the aforementioned texture memory and the requirement of low cost of the texture memory, it is difficult to enlarge capacity very greatly.
Therefore, the required texture data is stored in an external memory (hard disk, or the like) separate from the texture memory. By providing a time period during which no drawing is performed at times when the scene of the displayed image changes and by rewriting the texture data in a portion or the whole of the texture memory during this time period, it is apparently possible to handle a large amount of texture data.
FIGS. 1A-1E illustrate modes of storing MIPMAP texture data. FIGS. 1A-1E show, for example, texture data having five levels of detail (hereinafter, called LOD, for the sake of convenience). The horizontal axis is the x axis and the vertical axis is the y axis. FIG. 1A shows texture data having LOD=0, which is the highest level of detail. This data has the highest image resolution, and the size of the x axis in this data area is Size X, whilst the size of the y axis is Size Y.
FIG. 1B shows texture data having the next level of detail LOD=1; FIG. 1C shows texture data having the next level of detail LOD=2; FIG. 1D shows texture data having the fourth level of detail LOD=3; and FIG. 1E shows texture data having the lowest level of detail LOD=4. The texture data for LOD=1 to LOD=4 is derived by reducing the texture data for LOD=0. In other words, if the LOD value of the texture data rises to 1, then the x axis and y axis are reduced to xc2xd (surface area to xc2xc).
Therefore, the higher the level of detail, the lower the LOD value and the greater the amount of data, whereas the lower the level of detail, the higher the LOD value and the smaller the amount of data. Texture data of differing levels of detail, from LOD=0 to LOD=4, is used according to the size displayed on the screen. In other words, image processing is optimized in such a manner that, if the size displayed on the screen is large, then texture data of a higher level of detail is used, and if the size displayed on the screen is small, then texture data of a lower level of detail is used.
FIGS. 2A-2E show a example of a method for updating MIPMAP texture data as shown in FIGS. 1A-1E. In the example in FIGS. 2A-2E, the updating (rewriting) process is carried out in sequence, starting from the texture data of the highest level of detail (LOD=0). FIG. 2A shows texture data 0 having LOD=0, which is the highest level of detail, FIG. 2B shows texture data 1 having LOD=1, FIG. 2C shows texture data 2 having LOD=2, FIG. 2D shows texture data 3 having LOD=3, and FIG. 2E shows texture data 4 having LOD=4, which is the lowest level of detail.
The texture data 0 having the highest level of detail has the largest volume of data, and therefore it has the longest updating time, but the image displayed thereby is the most clear. Thereupon, as the updating process proceeds through texture data 1, texture data 2, texture data 3, texture data 4, the volume of data becomes smaller and the data volume of texture data 4 is the smallest. Consequently, texture data 4 having the lowest level of detail has the shortest updating time period, but the image displayed thereby is the least distinct.
Here, the area of the texture data 0 where the texture data has been updated is taken as the updated texture data area 11 indicated by the diagonal lines, and the area where texture data has not yet been updated is taken as the non-updated texture data area 12. In this case, the data at texel 21, a texel being the picture element unit for texture data, is data for which updating has been completed, but the data at texel 22 is data which has not been updated. Therefore, since the displayed image contains a mixture of updated texture data and non-updated texture data, the image will be distorted. However, since this occurrence is undesirable, usually, texture data is used in image processing for a new scene only after it has been confirmed that updating for all texture data from texture data 0 to 4 has been completed.
Nevertheless, in cases where scenes change rapidly with the passage of time, such as in game apparatuses, there has been a problem in that the fact of having to wait for updating of MIPMAP-format texture data to be completed has interrupted the flow of scenes.
Moreover, although it is possible to regulate the volume of texture data to a smaller data volume, in order that texture data can be updated within a prescribed period of time in such a manner that the flow of scenes is not interrupted, in this case, a problem arises in that insufficient usable texture data can be created.
Therefore, it is an object of the present invention to provide an image processing apparatus whereby distortion of displayed images can be suppressed, as far as possible, in cases where updating of texture data is not completed in time when displaying a prescribed scene.
In order to achieve the aforementioned object the present invention conducts update processing of MIPMAP-format texture data within an updateable range, and during drawing processing, it ascertains the level of detail to which texture data has been updated. Thereupon, when conducting texture mapping, texture data of the level of detail for which updating has been completed is used. In this case, if the required texture data has not been updated, then it is substituted by texture data for which updating has been completed. Consequently, even in cases where updating of all texture data is not completed in time, it is possible to prevent significant distortion of the image due to mixing of updated and non-updated texture.
In order to achieve the aforementioned object, the present invention is An image processing apparatus for processing drawing pixel data by using texture data corresponding to a plurality of levels of detail, comprising:
a rendering unit for calculating the level of detail of the pixel data to be drawn; and
a texture processing unit for updating at least a portion of the texture data corresponding to the plurality of levels of detail and for processing the texture data by using the level of detail of the updated texture data instead of calculated level of detail, in case that the texture data corresponding to the calculated level of detail has not yet been updated when drawing the pixel data.
In order to achieve the aforementioned object, for a more appropriate aspect of the present invention, the texture processing unit of the aforementioned image processing unit updates the texture data corresponding to the plurality of levels of detail in sequence starting from the lowest level of detail, and compares the level of detail of the updated texture data with the calculated level of detail, and processes the texture data by using the highest level of detail of the updated texture data in case that the calculated level of detail is higher than the highest level of detail of the updated texture data.
According to the present invention described above, it is possible to display images without significant distortion, even in cases where updating of texture data is not completed in time, when displaying a particular scene.
In order to achieve the aforementioned object, the present invention provides an image processing method for processing drawing pixel data by using texture data corresponding to a plurality of levels of detail, comprising the steps of:
calculating the level of detail of the pixel data to be drawn;
updating at least a portion of the texture data corresponding to the plurality of levels of detail; and
processing the texture data by using the level of detail of the updated texture data instead of calculated level of detail, in case that the texture data corresponding to the calculated level of detail has not yet been updated when drawing the pixel data.
In order to achieve the aforementioned object, the present invention provides a recording medium storing a program for causing a computer to implement an image processing method for processing drawing pixel data by using texture data corresponding to a plurality of levels of detail, the image processing method comprising the steps of:
updating at least a portion of the texture data corresponding to the plurality of levels of detail; and
processing the texture data by using the level of detail of the updated texture data instead of the level of detail of the pixel data.